Section: New Results

Matching and localization

Participants : Marie-Odile Berger, Antoine Fond, Pierre Rolin, Gilles Simon, Frédéric Sur.

Pose initialization

Estimating the pose of a camera from a model of the scene is a challenging problem when the camera is in a position not covered by the views used to build the model, because feature matching is difficult in such a situation. Several viewpoint simulation techniques have been recently proposed in this context. They generally come with a high computational cost, are limited to specific scenes such as urban environments or object-centered scenes, or need an initial guess for the pose. In [24], we have proposed a viewpoint simulation method well suited to most scenes and query views. Two major problems have been addressed: the positioning of the virtual viewpoints with respect to the scene, and the synthesis of geometrically consistent patches. Experimental results showed that patch synthesis dramatically improves the accuracy of the pose in case of difficult registration, with a limited computational cost.

Facade detection and matching

Planar building facades are semantically meaningful city-scale landmarks. Such landmarks are essential for localization and guidance tasks in GPS-denied areas which are common in urban environments. Detection of facades is also key in augmented reality systems that allow for the annotation of prominent features in the user's view. We introduced several “facadeness” measures of image regions and showed how to combine them to generate building facade proposals in images of urban environments [26]. We demonstrated the interest of this procedure through two applications. First, a convolutional neural network (CNN) was proposed to detect facades from a restricted list of facade proposals. We showed that this method outperforms the state-of-the-art techniques in term of adequation of the detected facades with a ground truth. In addition, the computational time is compatible with the navigation requirements. Second, we investigated image matching based on facade proposals. Considering a large set of data extracted from Google Street View, we showed that matching based on Euclidean distances between CNN descriptors outperforms the classical SIFT matching based on RANSAC-homography calculation. This work has been submitted to IEEE ICRA'2017.

A preliminary step in facade detection is the image rectification process. For that purpose, we introduced a simple and effective method to detect orthogonal vanishing points in Manhattan scenes. A key element of this approach is to explicitly detect the horizon line before detecting the vanishing points, which is done by exploiting accumulations of oriented segments around the horizon line. This results in a significant reduction in computation times, while keeping an accuracy comparable or superior to more complex approaches. A paper reporting on this work was published and an oral presentation was made at Eurographics'2016 [25].